STM Studies of TbTe3i Evidence for a fully Incommensurate Charge Density Wave 
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We observe unidirectional charge density wave ordering on the cleaved surface of TbTe3 with 
a Scanning Tunneling Microscope at ~6 K. The modulation wave- vector qcDW as determined by 
Fourier analysis is 0.71 =b 0.02 x2tt/c. (Where c is one edge of the in-plane 3D unit cell.) Images 
at different tip-sample voltages show the unit cell doubling effects of dimerization and the layer 
below. Our results agree with bulk X-ray measurements, with the addition of (1/3) x27r/a ordering 
perpendicular to the CDW. Our analysis indicates that the CDW is incommensurate. 

PACS numbers: 71.45.Lr, 61.44.Fw, 68.37.Ef, 72.15.-v 



Charge density waves (CDW) in weakly interacting 
systems are predominantly driven by Fermi surface (FS) 
nesting, where a single vector qcDW = 2fc/ at the peak 
of the generalized susceptibility connects points on the 
FS. In 2D systems, this nesting is enhanced if the FS has 
parallel regions. The rare-earth tri-telluride series RTes 
(R = rare-earth) is such a quasi-2D system, formed in 
the orthorhombic space group Cmcm [l|, and contains 
double layers of nominally square Te planes separated by 
RTe block layers. A unidirectional CDW [2] was first de- 
tected in this system by transmission electron microscopy 
(TEM) [3] . Further measurements of angle resolved pho- 
toemission spectroscopy (ARPES) have shown that large 
portions of the FS nested by qcDW are indeed gapped, 
indicating that the CDW is driven by FS nesting 0, [H[ . 

The band structure near the Fermi surface can be mod- 
eled using the Te p x and p y in-plane bonding orbitals in 
a single Te plane, which is doped from the block layer. 
Such a model has tetragonal C4 symmetry and a Bril- 
louin zone, hereafter referred to as the 2DBZ. This model 
fits the measured bands well [4( and theoretical investiga- 
tions show a nesting along the diagonal of the 2DBZ, 
with qcDW ~ (0.75,0.75) x 7r/ao, where ao is the Te- 
Te spacing of 3.1 A. Yao et al [6] argued that this 
model has a hidden ID nature, resulting in a unidirec- 
tional CDW. The structural modulation arising from the 
CDW in .RTes has also been observed by X-Ray Diffrac- 
tion (XRD) @, H, @|. Though the CDW exists in the 
Te planes, the adjacent block layer introduces another 
super-lattice with in-plane basis vectors (a, c) that are 
45° to the Te net and times longer than ao-[10] The 
new reduced three-dimensional Brillouin zone (3DBZ) 
has half the in-plane area and is rotated 45° from the 
2DBZ. This additional periodicity folds (a small amount 
of) spectral weight from the bands back into the 3DBZ. 
For diffraction measurements such as XRD, the wave- 
vector is ambiguous within a reciprocal lattice vector, 
and thus is often reported inside the reduced 3DBZ as 
qcDW ~ 2/7 x 27r/c. On the other hand, ARPES re- 
ports qcDW ~ 5/7 x 27r/c based on gaps in the FS. 
Thus there exists some ambiguity regarding which mod- 



ulation is the dominant contribution to the CDW. Re- 
garding the structure of the CDW, Kim et al. in a recent 
paper employed atomic pair distribution function (PDF) 
analysis of powder X-Ray data supplemented with local 
STM measurements and suggested that the modulation 
in CeTe3 is composed of commensurate domains sepa- 
rated by discommensurations. [ll| 
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FIG. 1: a) Topography taken with a voltage bias of +200 mV, 
50 pA current, b) Average spectrum in the range ±800 mV. 
Arrows indicate kinks in the spectra near the gap edge. 

In this paper, we use scanning tunneling microscopy 
and spectroscopy to study the unidirectional CDW sys- 
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tern TbTe3. Large area scans using a range of bias volt- 
ages allow us to resolve the ambiguity regarding the wave- 
vectors that comprise the CDW and show evidence of its 
fully incommensurate nature. Topographic data at dif- 
ferent bias voltages highlight two spontaneous symmetry 
breaking effects in breaking the lattice point group sym- 
metry: formation of the CDW and dimerization. 

We performed measurements on a home-made UHV 
cryogenic STM. Single crystals were grown using a self- 
flux method as previously reported |l2[. The samples 
were cleaved between the two Te planes in < 2 x 10 -10 
Torr vacuum and quickly lowered to the ~6 K section 
of the microscope, where cryo-pumping ensures that the 
surface remains free from adsorbates. Topography were 
taken at several bias voltages (= V sarnp i e — Vu p ) and set- 
point currents (typically lOOpA or less). Scan sizes were 
as large as ~ 240 x 240 A 2 . While surfaces often have 
large areas with no obvious surface impurities (which 
might pin the CDW), flakes or other "dirt" every few 
hundred A limits the maximum size of our scans. Ther- 
mal compensation, hysteresis minimization techniques, 
and post processing were used to reduce the amount of 
scan distortion in the images. An example of such a scan 
with bias voltage +200 mV is shown in Fig. We also 
took spectroscopic scans over smaller areas, i.e. a dl/dV 
(proportional to the local density of states) spectrum at 
every point. The spectra mainly had spatial variations 
relating to the lattice and CDW, thus we show in Fig. [TJd 
an averaged spectrum taken in the range of ± 800 mV. 
The CDW gap is marked with arrows. States in the gap 
are clearly seen, indicating that the FS is only partially 
gapped. The maximal gap was measured at « 240 mV 
by photoemission. [l3[ 

For STM, CDWs should appear in topography. How- 
ever, different sample bias may give different results de- 
pending on how many of the states responsible for the 
CDW are integrated. Fig. [2^ shows a (zoomed in) scan 
at -800mV and 50 pA, a voltage outside the gap which 
should include all the states responsible for CDW for- 
mation. We observe the square lattice of the surface 
Te layer, with an observed Te-Te (average) spacing of 
« 3 A, in good agreement with bulk diffraction mea- 
surements [8]. The atomic features visually swamp any 
CDW modulation. One possible explanation for the rel- 
atively small amplitude of the CDW signal is that at 
these large negative energies, (and relatively low tun- 
nel currents) we are probing the filled p z orbitals of the 
Te atoms. Their extended nature causes the tip to be 
further away from where the CDW wavefunctions exist. 
Since the CDW wave function decays exponentially out 
of the plane [3] , its contribution to the topography is ex- 
pected to be small. The second is that it is energetically 
unfavorable to have large charge inhomogeneities due to 
Coulomb repulsion, and thus when looking at quantities 
that are closer to representing total charge, i.e. large bias 
voltages, the CDW amplitude appears small. 
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FIG. 2: Zoomed-in view of topographic scans with 50 pA 
setpoint current. All units in A. a) Topography at -800mV b) 
-200mV. Right half shows locations of the surface Te atoms, 
c) +200mV. 



We also note the presence of dimerization, with pairs 
of atoms connected in an upper- left, lower-right direc- 
tion. First, this represents another broken symmetry, as 
it chooses a direction 45° (as opposed to -45°) to the 
CDW. Second, this is another mechanism for the unit 
cell doubling seen in this material. Although the Te net 
is expected to be unstable against a 3.1 A bond length 
[l5j |. this effect has not been directly observed by XRD, 
but has been inferred in the PDF performed in ref [Tlj . 
We do not see a complex pattern of oligimers as proposed 
in the works of Malliakas et al. [8] since the simple two- 
atom dimers have a repeating pattern of long and short 
bonds within a row. This is also suggested in Fig. [3jo 
(red line) by a lack of lower frequency components in the 
FFT. Although what we see may only be a surface effect, 
this observation should prompt a re-examination of the 
RTes crystal structure above and below T c . 

Fig. [2}d is data taken over the same area at -200mV, 
a value inside the maximal CDW gap. Since the Fermi 
surface is only partially gapped, we expect to see states 
responsible for CDW formation at these lower energies. 
We note again a unit cell doubling effect which shows 
up not as dimers, but as a new square lattice rotated 
45°. With respect to the surface Te plane, three different 
mechanisms for unit cell doubling have been proposed in 
RTes: Dimerization, the adjacent block layer, and the 
stacking of the pair of Te layers deeper in the sample, 
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which results in two crystallographically inequivalent Te 
atoms at the surface. If measured with a real space probe, 
these mechanisms all give the same Fourier points for the 
superlattice, albeit with different registry to the surface 
Te atoms. The peaks of the superlattice would fall be- 
tween two Te atoms, four Te atoms, or on every other 
Te atom, respectively. Since we lack spatial registry be- 
tween scans, we instead look at the phase of the Fourier 
peaks for the Te lattice combined with the spectroscopic 
data. The locations of the surface Te atoms are depicted 
as white circles in the right half of Fig. [2)3. Since the red 
spots fall between four Te atoms, this suggests that we 
are imaging effects from the Tb atoms in the block layer. 
Seeing effects from the next layer below is not unheard 
of in STM, e.g. graphite [16]. For this set of data, the 
tip is 3 A closer to the sample than it was at -800mV 
bias. Thus, the CDW and the unit cell doubling effects 
from the block layer show up more strongly. 

Fig. [2t shows a zoomed-in topograph at +200 mV. 
According to the spectra, the lower density of states at 
positive bias suggests that the tip is even closer to the 
sample. The irregular shapes of the "atoms" is likely due 
to the convolution of the true sample topography with tip 
states. This set of data shows most strongly the CDW 
ordering and effects of the block layer. 

To understand the structure of the CDW, we compare 
the Fourier transforms of the -f 200mV and -800mV to- 
pographies. In Fig. [3^l (+200mV) the square lattice from 
the Te atoms is clearly seen, as well as that of the block 
layer, which creates the unit-cell doubling effect as de- 
scribed earlier. Fig. 03d (blue line) is a line cut showing 
five peaks- the superlattice at 27r/c, and four interme- 
diary peaks at 0.29, 0.42, 0.58, and .71 ±0.02 x 2tt/c. 
In another interpretation, Kim et al.[ll| identified these 
peaks as satellites from a discommensuration envelope 
[191 ]. However, the -800 mV data (Fig. [3jo red line) show 
that qcDW=®-71, a value outside the 3DBZ, is the domi- 
nant contribution to the CDW, and that the other peaks 
are greatly reduced. Thus they are not representative 
of an intrinsically discommensurate structure, as they do 
not exist under different tunneling conditions. Rather, 
we identify them as 0.29=l-g, 0.42=2<?-l, and 0.58=2-2<? 
which is mixing between q and the superlattice. These 
peaks result from distortions of STM DOS measurements 
due to the underlying block layer. This distortion seems 
to increase when the superlattice point is stronger, as 
in the blue line. It is analogous to the effect seen in 
ARPES [1] where the strength of coupling to the super- 
lattice period determines the amount of spectral weight 
folded back. In our data, the Fourier peaks are narrow, 
indicating a long coherence length for the modulations. 
Additional width to the peaks comes from a combination 
of scan distortion, spectral leakage [17], and some varia- 
tion in the CDW amplitude, possibly due to defects below 
the surface. Overall, our interpretation that the "true" 
CDW has little harmonic content and a long coherence 
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FIG. 3: (Color) (a) FFT of +200 mV topography (Fig.^a)). 
O = Origin; E = Te square lattice (2DBZ); R = block layer 
(3DBZ). (b) Blue: Line cut from O to R, thru the CDW. Red: 
same analysis for -800 mV topograph. Amplitude rescaled for 
clarity, (c) Line cut from the CDW point C to D. 



length is consistent with the X-Ray data[18[ which show 
a nearly perfectly sinusoidal modulation in the bulk. 

Having ruled out a nearly commensurate (NC) struc- 
ture, next we check if the CDW is fully incommensu- 
rate or commensurate against the nearest fraction of 5/7 
(=0.714). (Although it is difficult to distinguish incom- 
mensurate against a high denominator fraction commen- 
surate.) Recent X-ray data shows that the CDW is in- 
commensurate in the bulk at room temperature with 
4=0.704 ±0.0010. However, it is well known that in- 
commensurate CDW materials can change their wave- 
vectors as a function of temperature [9] or even undergo a 
change to the NC or commensurate structure [19| . There 
may also be surface effects which make the CDW differ- 
ent at the surface than the bulk. [20j] 

We check our low temperature data for commensura- 
tion against 5/7 by filtering out features perpendicular 
to the CDW wave- vector, then taking a line cut in the 
direction of the CDW wave- vector in real space. We then 
match up groups of 7 unit cells. If the CDW were com- 
mensurate, all the groups would be identical. Fig. 2^ 
shows the results of this analysis, with the arrow point- 
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FIG. 4: (Color) (a) Line scan along the CDW, in groups of 7 
unit cells. Blue is one end of the sample, evolving to dark red 
(other end) (b) Same signal, with atomic features filtered out, 
and amplitude multiplied by 2. (c) Main CDW component 
only. 



X-ray diffraction, meaning that it is either a previously 
unobserved second CDW phase transition below room 
temperature or a surface reconstruction. 

In conclusion, we show that TbTe3 has a fully in- 
commensurate, unidirectional CDW at low temperatures, 
with a dominant wave vector q=0.71 ±0.02 outside the 
3DBZ. The adjacent R-Te block layer strongly affects the 
topography at lower bias voltages by acting as an addi- 
tional periodicity which distorts STM measurements of 
the CDW. Our observation of simple two-atom dimeriza- 
tion in the Te net is consistent with the bimodal bond 
length distribution suggested in powder PDF, and our 
interpretation of the CDW as fully incommensurate with 
long coherence length agrees with X-Ray data. 
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ing at a region of smooth evolution from one group to 
the next. From this, we conclude that the CDW is in- 
commensurate. Fig. [4}d is the same data, with the lat- 
tice filtered out. A ~5/7 modulation can clearly be 
seen, with some distortion caused by the lattice positions. 
This distortion creates the other peaks in Fig. [3)3 (blue 
line). Fig. [It shows the Fourier filtered component for the 
CDW slowly advancing as expected for slight incommen- 
suration with q < 5/7. [21] This value of q means that the 
four peaks in Fig. [3](b) (blue line) are not evenly spaced, 
but rather the two center peaks are slightly spread apart. 
Likewise, in the work of Kim et al,[ll| the two center 
peaks are slightly closer together, consistent with q >5/7 
in CeTe 3 . 

It is not unexpected that this material would be incom- 
mensurate for the following two reasons: 1) The CDW is 
FS nesting driven, and thus does not have to be related 
to the lattice by a rational fraction. 2) The high order 
denominator (> 7) for the nearest commensuration frac- 
tion implies that any lattice locking or commensuration 
effects are weak.[22j In addition, resistivity and heat ca- 
pacity measurements do not indicate any transitions from 
room temperature down to 1.8 K (aside from a Neel tran- 
sition at 5.6 K). [12] Thus we conclude that for R=Tb, 
the CDW stays incommensurate with q < 5/7 down to 
low temperatures. 

Finally, we note weak Fourier signals at q c ^(0.29 or 
0.71) and q a ^(0.32 or 0.68), as satellites perpendicular 
to the CDW. Thus, in Fig. [3k we take a like cut in the 
direction from C to D. This means that there is a ~ 
3 x a modulation perpendicular to the CDW, and that 
each row of the CDW is shifted laterally with respect to 
the next. This signal is not seen in room temperature 
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